Autonomous vehicles are paving way for a new mode of transportation. Autonomous vehicles require minimum or no intervention from vehicle's driver. Generally, some autonomous vehicles need only an initial input from the driver, whereas some other designs of the autonomous vehicles are continuously under control of the driver. There are some autonomous vehicles that can be remotely controlled. For example, automatic parking in vehicles is an example of the autonomous vehicle in operation.
Autonomous vehicles face a dynamic environment that is the environment keeps changing every time. The autonomous vehicles need to keep a track of lane markings, road edges, track road curves, varying surfaces that may include flat surfaces, winding roads, hilly roads etc. Alongside, the autonomous vehicles also need to keep a check on objects that are both stationary or mobile like a tree or a human or an animal. Hence, the autonomous vehicles need to capture a huge amount of information that keeps on changing every time.
Therefore, to overcome and meet these challenges, autonomous vehicles are provided with a various set of sensors. These sensors help the vehicle to gather all around information and help in increasing the degree of autonomy of the vehicle. The various types of sensors currently being used in autonomous vehicles are LiDAR sensors, Ultrasonic sensors, Image sensors, Global Positioning System (GPS) sensors, Inertial Measurement Unit (IMU) sensors, dead reckoning sensors, Microbolo sensors, Speed sensors, Steering-angle sensors, Rotational speed sensors, Real-time Kinematics sensors, and RADAR sensors. Two of the most used sensors are LiDAR and RADAR sensors.
LiDAR sensors: LiDAR is a device that maps objects in 3-dimensional by bouncing laser beams off its real-world surroundings. LiDAR in automotive systems typically uses a 905 nm wavelength that can provide up to 200 m range in restricted FOVs (field of views). These sensors scan the environment, around the vehicle, with a non-visible laser beam. LiDAR sensor continually fires off beams of laser light, and then measures how long it takes for the light to return to the sensor. The laser beam generated is of low intensity and non-harmful. The beam visualizes objects and measures ranges to create a 3D image of the vehicle's surrounding environment. LiDAR sensors are very accurate and can gather information to even up to very close distances around the vehicle. However, LiDAR sensors are generally bulky, complex in design and expensive to use. The costs can be between around $8,000 and even up to $100,000. Smaller and less expensive LiDAR sensors are starting to be in the market. LiDAR may also require complex computing of the data collected that also adds to the costs. Also, in general, LiDARs can capture data up to a distance of around 200 m.
It is to be noted that LiDAR requires optical filters to remove sensitivity to ambient light and to prevent spoofing from other LiDARs. Also, the laser technology used has to be “eye-safe”. Recently there are efforts being made to replace mechanical scanning LiDAR, that physically rotates the laser and receiver assembly to collect data over an area that spans up to 360° with Solid State LiDAR (SSL). SSLs have no moving parts and are therefore more reliable especially in an automotive environment that requires long-term reliability. However, SSLs currently have lower field-of-view (FOV) coverage.
In current LiDAR sensor design coverage is also a problem in terms of sensor gap and overlap, since the LiDAR in autonomous vehicles has very limited redundancy sensors that can provide the level of imaging that a LiDAR provides under optimal conditions. LiDAR is also weather susceptible. It turns blind when it comes to imaging in adverse weather conditions. LiDAR has limitations of creating clear imaging in conditions of fog, rain, snow, direct sunlight, and darkness. Also, LiDAR cannot read letters on a signboard. This is so because the signboard is flat.
RADAR sensors: RADAR sensors basically send out electromagnetic waves. When these waves hit an obstacle, they get reflected. Thus, revealing how far away an object is and how fast is it moving.
RADAR sensors are very crucial in today's autonomous vehicle applications. They are required to be more accurate.
Automotive RADARs can be categorized into three types: long-range RADARs, medium range RADARs, and short-range RADARs. Long range RADARs are used for measuring the distance to and speed of other vehicles. Medium range RADARs are used for detecting objects within a wider field of view e.g. for cross traffic alert systems. Short range RADARs are used for sensing in the vicinity of the car, e.g. for parking aid or obstacle detection. Depending on the application, RADAR requirements differ. Short range applications require a steerable antenna with a large scanning angle, creating a wide field of view. Long range applications, on the other hand, require more directive antennas that provide a higher resolution within a more limited scanning range. Two different frequency bands are mainly used for automotive RADARs: the 24 GHz band and the 77 GHz band. The 77 GHz band offers higher performance, but it is also more challenging to implement since for example losses are much higher at these frequencies. The 24 GHz RADARs are easier to develop but are larger in size, making it difficult to integrate them in a vehicle. RADARs operating at 24 GHz require around three times larger antennas than RADARs operating at 77 GHz, to achieve the same performance. A 77 GHz RADAR would thus be much smaller resulting in easier integration and lower cost. Moving to higher frequencies enables RADARs with a better resolution. However, a major challenge posed is to develop steerable antennas for 77 GHz RADARs with high enough performance at a reasonable cost. In one embodiment of the invention different types of antenna and meta-material-based antennas that are less prone to phase noise disturbance will benefit from this invention.
Some Automotive RADAR systems use a pulse-Doppler approach, where the transmitter operates for a short period, known as the pulse repetition interval (PRI), then the system switches to receive mode until the next transmit pulse. As the RADAR signal returns, the reflections are processed coherently to extract range and relative motion of detected objects. Another approach is to use Continuous Wave Frequency Modulation (CWFM) or Frequency Modulated Continuous Wave (FMCW). This approach uses a continuous carrier frequency that varies over time with a receiver constantly on. To prevent the transmit signal from leaking into the receiver, separate transmit and receive antennas are used.
Generally, there are three types of RADARs in use in autonomous vehicles. Short range RADAR, that helps in collision warning, and provide assisted parking support. Medium Range RADAR helps to watch corners of the vehicle, help in blind spot detection, lane detection and avoid side/corner collisions. Further, long-range RADARs help in adaptive cruise control functions and early collision detection functions.
RADAR signal processing also needs to be efficient. It needs to intelligently group the bouncing signals from the same object in the range. Otherwise, the RADAR signal processing will be overwhelmed with the amount of signal processing and may get confused. Grouping is made possible by use of Doppler shift of the signals bouncing off from the surfaces with a. velocity different from the observation domain. Thus, Doppler maps are created, that depict the range to object returns on one axis and extracted velocity of the targets on the other.
RADARs have also been used for identifying and classifying humans. RADARs are not efficient in performing the human identification; however, there have been techniques that are being used for human identification.
One of the techniques uses reflectivity of humans using an ultra-wideband RADAR. In this technique, the polarization of the reflected signal can be analyzed and can be determined that there are some frequencies where, in one polarization, there is maximum reflectivity and a minimum in the other. However, the polarized signal depends mainly on the shape, posture, and position of the person. Thus, making it a highly unreliable technique for classification.
Another technique used for human classification uses a dual-band frequency modulated continuous wave RADAR. In this technique, the difference in reflected signal from an object at different frequencies (commonly 10 Hz to 66 Hz) is compared. Through this comparison, the threshold for the ratio of the received intensity between two frequencies, above which detected objects can be classified as animated was established.
Some techniques utilized and analyzed Doppler spectrum of CW RADAR to obtain a Doppler or micro-Doppler Signature for a walking human. Whereas, some of the techniques used wavelet transform to extract the micro signatures created by human walking. The same techniques can be utilized for other human movement and gesture control.
Other techniques that should be mentioned here include the use of 2 or more different frequencies and multiple chirp types, These and the techniques above support the evaluation and recognition of the electromagnetic characteristics and properties of a human being or another targeted object. These techniques can be used for material detection and human classification. Also, the usage of multiple frequencies and chirp types provides information for micro-Doppler and radar signature evaluation and recognition.
The techniques mentioned are directed towards identification of a human and distinguish them from other walking objects like animals. However, these techniques can also be used to recognize animals.
RADAR sensors are low priced and provide as excellent sensors. RADARs also cost much less than LiDAR and may be procured within $150. These sensors work extremely accurately in bad weather conditions like fog, snow, dirt, etc. RADAR sensors use extremely simple circuitry and thus are smaller in size that makes them easy to be manufactured, installed and used. However, one of the major drawbacks of the RADAR sensors is that they give confusing results when multiple objects are within the range. They are not able to filter noise in such situations. Existing RADARs do not offer the necessary resolution to distinguish objects with sufficient reliability. One of the main problems faced is the separation of small and large objects that travel at the same distance and velocity in adjacent lanes, e.g. a motorcycle driving in the lane next to a truck.
Major factors affecting RADAR performance are described in the following paragraphs:
Transmitter Power and Antenna:
The maximum range of a RADAR system depends in large part on the average power of its transmitter and the physical size of its antenna. This is also called the power-aperture product. The antenna itself remains a challenge for autonomous vehicles and a lot is invested in various antenna developments. The invention described here can be used with a type of antenna including antennas based and made out of meta-materials. In fact, the synergy between a meta-material antenna and this invention would result in a very high performing radar sensor.
Receiver Noise:
The sensitivity of a RADAR receiver is determined by the unavoidable noise that appears at its input. At microwave RADAR frequencies, the noise that limits detectability is usually generated by the receiver itself (i.e., by the random motion of electrons at the input of the receiver) rather than by external noise that enters the receiver via the antenna.
Target Size:
The size of a target as “seen” by RADAR is not always related to the physical size of the object. The measure of the target size as observed by RADAR is called RADAR cross-section and is determined in units of area (square meters). It is possible for two targets with the same physical cross-sectional area to differ considerably in RADAR size or RADAR cross-section. For example, a flat plate 1 square meter in the area will produce a RADAR cross-section of about 1,000 square meters at a frequency of 3 GHz when viewed perpendicular to the surface. A cone-sphere (an object resembling an ice-cream cone) when viewed in the direction of the cone rather than the sphere could have a RADAR cross-section of about 0.001 square meters even though its projected area is also 1 square meter. Hence, this may cause calculation mistakes and may give the wrong estimation of the objects identified.
Clutter:
Echoes from environmental factors like land, rain, birds and other similar objects may cause a nuisance to detect objects. Clutter makes it difficult to identify objects and their properties to a considerable extent.
Interference:
Signals from nearby RADARs and other transmitters can be strong enough to enter a RADAR receiver and produce spurious responses. Interference is not as easily ignored by automatic detection and tracking systems. Hence, interference may further add to noise to the RADAR signals.
Phase-Noise:
Phase-noise is defined as the noise created by short term phase fluctuations that occur in a signal. The fluctuations display themselves in the frequency domain as sidebands which appear as a noise spectrum spreading out either side of the signal (Can be seen on FIG. 34)
Comparison Between LiDAR and RADAR
As compared to LiDAR sensors, RADAR sensors provide more robust information to the vehicles. LiDAR sensors are generally mounted on top of the vehicle and are mechanically rotated to gather surrounding information. This rotational movement is prone to dysfunction. Whereas in case of RADAR, as they are solid state and have no moving parts hence have a minimal rate of failures.
Also, LiDAR sensors produce pulsed laser beams and hence are able to gather information only when the pulsed beam is generating the laser beams. RADAR sensors can generate continuous beams and hence provide continuous information.
Also, LiDAR sensors generate enormous and complex data for which complex computational modules are required to be used. For example, some types of LiDAR systems generate amounts of 1-Gb/s data that require a substantial amount of computation by strong computers to process such high amount of data in a timely manner. In some cases, these massive computations require additional computation and correlation of information from other sensors and sources of information. These increases cost overheads for vehicle manufacturers. Whereas, RADAR sensors only generate small fractions of data that are easy to compute.
LiDAR sensors are also sensitive to adverse weather conditions such as rain, fog, and snow while RADAR sensors are not prone to weather conditions. Though RADAR is not affected by darkness and it can work well in adverse weather conditions, it may lower its resolution.
RADAR signatures of a walking human being are a big problem. These signatures are not easily recognizable. Detection of human beings is a problem to which most of the computing algorithms do not have good solutions. There are various algorithms known as segmenting algorithms that do provide a certain level of the solution. The processing engine of an autonomous vehicle may take input from RADAR, LiDAR, Camera, Ultrasound and other multiple sensors to build an image of the surroundings of the vehicle.
Further, RADARs are generally used to detect receding and approaching objects. Use of RADARs helps to decelerate the vehicle in applicable situations and warn the driver.
Stationary RADAR sensors are also in use to monitor a predetermined space for e.g. railway crossings may be monitored using stationary RADARs. The usage includes identification of objects in such railway crossings. In such situations either a warning can be generated, or the train may be decelerated. However, to effectively use the stationary RADARs it is very important to be able to determine the size of the objects identified by the RADAR sensor. These RADARs may include height estimating systems for objects located in the range of such a RADAR. This needs to be accurate as even small amounts of deviations between the plane and vertical sensor axis can result in large errors in estimating the object's size.
However, RADAR sensors are challenged when dealing with slow-moving objects such as cars, bicycles, and pedestrians. Furthermore, these traditional RADAR systems, whether using a modulated or non-modulated signal, have difficulties identifying objects that are very close to each other since one of them will be obscured due to the phase-noise of the system. Also, the drawback of existing RADAR sensors is the impact on their accuracy due to the phase-noise of its frequency source, the synthesizer. RADAR sensors are not able to relay size and shape of objects as accurately as LiDAR. RADAR sensors might not be a stand-alone solution. They can be accompanied by ultrasonic sensors or cameras. Though, RADAR is excellent in finding things that are solid over long distances, but, it may be challenging to identify things that are in a short range.
Therefore, there is a need for an enhanced detection system capable of implementing artificial intelligence using various sensory fusion including a plurality of LiDAR, Camera, Ultrasound, and RADAR sensors for helping in making informed decisions based on surrounding information for semi or autonomous vehicles. Furthermore, the system should be capable to overcome the shortcomings of the existing systems and technologies.